Skip to content

Watching a statement while it runs - #22

Merged
tamnd merged 1 commit into
mainfrom
dx-progress
Aug 19, 2026
Merged

Watching a statement while it runs#22
tamnd merged 1 commit into
mainfrom
dx-progress

Conversation

@tamnd

@tamnd tamnd commented Aug 19, 2026

Copy link
Copy Markdown
Owner

rowsRead is how far the statement running now has got, and progress is a timer around it. A statement that takes a minute is one somebody is sitting in front of, and this client had nothing to tell them while it ran.

using watch = conn.progress((rows) => process.stdout.write(`\r${rows} rows read`));
const answer = await conn.query(statement);

The count is read the way open is, as an atomic beside the connection's lock rather than a question through it, so it answers on the loop's thread while the statement holds the lock on a threadpool thread. It counts rows read out of storage rather than rows answered, because the statement somebody is waiting on is exactly the one that reads a hundred million rows to answer one.

The watch itself is JavaScript, in zudb.cjs beside stream. A timer is all it takes, since the loop is free while the statement runs, and nothing calls into JavaScript from the thread doing the scanning: a thread safe function there would be a scan that stops to talk to the loop. The callback runs only when the count has moved, so a watch on an idle connection says nothing and one nobody stopped is not a callback ten times a second forever, and the timer does not hold the process open. It is stopped by stop() or by leaving the scope of the using, and the interval is { everyMs } with a tenth of a second by default.

One thing turned up on the way. The counter was never put back down between statements here, so it had been counting a connection's whole life rather than one statement, which is neither what Python does nor what the shell does. It is cleared now where the connection becomes a statement's, which is the one moment that can be done without racing the getter, and the three places that happens are the query, the stream and the profile. So conn.rowsRead after a statement is what that statement cost.

Ten tests, including that the number moves during the statement rather than after it, that a watch on an idle connection stays quiet, that a stopped one stops, that a using stops it at the end of the block, and that a program with a watch nobody stopped still exits. The whole suite is green locally, with the types, the API report and the formatting.

rowsRead is how far the statement running now has got, and progress is a
timer around it. A statement that takes a minute is one somebody is
sitting in front of, and this client had nothing to tell them while it
ran.

The count is read the way `open` is, as an atomic beside the
connection's lock rather than a question through it, so it answers on
the loop's thread while the statement holds the lock on a threadpool
thread. It counts rows read out of storage rather than rows answered,
because the statement somebody is waiting on is exactly the one that
reads a hundred million rows to answer one.

The watch itself is JavaScript, in zudb.cjs beside stream. A timer is
all it takes, since the loop is free while the statement runs, and
nothing calls into JavaScript from the thread doing the scanning: a
thread safe function there would be a scan that stops to talk to the
loop. The callback runs only when the count has moved, so a watch on an
idle connection says nothing and one nobody stopped is not a callback
ten times a second forever, and the timer does not hold the process
open.

One thing turned up on the way. The counter was never put back down
between statements here, so it had been counting a connection's whole
life rather than one statement, which is neither what Python does nor
what the shell does. It is cleared now where the connection becomes a
statement's, which is the one moment that can be done without racing the
getter, and the three places that happens are the query, the stream and
the profile.
@tamnd
tamnd merged commit 1f56c30 into main Aug 19, 2026
10 of 13 checks passed
@tamnd
tamnd deleted the dx-progress branch August 19, 2026 13:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant